1/1 


AD-A157  712 
UNCLASSIFIED 


VALIDATION  OF  THE  DATA  SOURCES  AND  PROCESSES  OF  THE 
VAHOSC  (VISIBILITV  AN.  .  (U)  INFORMATION  SPECTRUM  INC 
HUNTINGDON  VALLEV  PA  SJ  EINHORN  30  AUG  84 
ISI-V-84-31859-21  F33600-82-C-0543  F/G  9/2 


NL 


#  -  *3 


INFORMATION  SPECTRUM,  INC. 

3883  HUNTINGDON  PIKE  •  HUNTINGDON  VALLEY,  PA.  19008  •  (215)  947-6080  , 


VALIDATION  OF 

THE  DATA  SOURCES  AND  PROCESSES 
OF  THE  VAMOSC  II 
COMPONENT  SUPPORT  COST  SYSTEM 
(D160B) 


FINAL  REPORT 


•  Contract  No.  F33600-82-C-0543 


30  August  1984 


Submitted  to : 


Report  Number 
V- 84- 31 85 9-21 


Prepared  by: 


DTIC 

kELECTE| 
l  AU6  7  1985 


HEADQUARTERS 

AIR  FORCE  LOGISTICS  COMMAND 
MM (VAMOSC) 

WRIGHT-PATTERSON  AFB ,  OH  45433 


Dr.  Sheldorf'  J.  Einhorn 
Chief  Scientist 


This  document  has  been  approved 
lor  public  release  and  sale;  its 
distribution  is  unlimited 


1748  JEFFERSON  DAVIS  HIGHWAY 
ARLINGTON,  VIRGINIA  22202 
(703)882-8000 

A  -*» 


85  7  19 

k *  A  ~  k  m  *  •  A  *  •  •  *  *  n  *  «  •  «  ■  «_• 


1040  KINGS  HIGHWAY  NORTH 
A  -  CHERRY  HILL,  NEW  JERSEY  08034 

044  (800)867-6181 


TABLE  OF  CONTENTS 


Section 


1.0  INTRODUCTION 


1 


2.0  SUMMARY  OF  PROJECT  ACTIVITIES .  6 

2.1  Algorithm  Analyses  .  6 

2.2  NSN/WUC  Cross  Reference  Dictionary  .  9 

2.3  Base  Supply  Management  Overhead  Costs  .  9 

2.4  User  Survey  .  10 


3.0  RESULTS  .  13 

3.1  Recommendations  .  13 

3.1.1  Algorithms  . 13 

3.1.2  Cross-Reference  Dictionary  .  14 

3.2  Input  Data  System  Accuracy  . 15 

3.3  User  Survey  . 17 

3.4  CSCS  Output  Report  Accuracy  .  17 


APPENDIX  A  -  COMPILATION  OF  CRITICISMS,  RECOMMENDATIONS, 

AND  RESPONSES  FOR  CSCS  COST  ALGORITHMS  _  Al-1 


1.  Base  Direct  Labor  Rates . Al-1 

2.  Military  and  Civilian  Labor  at  Base  Level  .  Al-2 

3.  Inflation  Factors  .  Al-3 

4.  System  Documentation  . Al-3 

5.  Input  Data  System  Error  . Al-4 

6.  Labor  Rates  for  Overhead  Costs  .  Al-4 

7.  Depot  Activity  Rates  . . Al-5 

8.  Depot  Cos+-  Extrapolation  . Al-6 

9.  Depot  MDS  Labor  Rates  . Al-6 

10.  Depot  MDS  "Other”  Cost  Rates  . Al-7 

11.  Depot  MDS  Material  Rate . Al-8 

12.  Depot  Support  General  Other  Costs  .  Al-8 

13.  Base  Supply  Management  Overhead  .  Al-8 

14.  Shipping  Costs  . Al-9 

15.  Engine  Shipments  .  Al-10 

16.  Engine  Shipping  Costs  .  Al-11 

17.  Data  Processing  for  Depot  Material  Management 

Overhead  Costs  .  Al-12 

18.  Depot  Material  Management  Overhead  Factor  .  Al-12 

APPENDIX  B  -  SURVEY  ANALYSIS  .  Bl-1 


LIST  OP  TABLES 


Number 

1  CSCS  Algorithm  Names  . 

2  CSCS  Algorithm  Analysis  Reports 


LIST  OP  FIGURES 


Number 

1  CSCS  User  Survey 


>  *» 


I 


1.0  INTRODUCTION 

Visibility  and  Management  of  Operating  and  Support  Costs 
(VAMOSC)  is  a  program  initiated  by  the  Office  of  the  Secretary 
of  Defense  (OSD)  in  order  to  ensure  that  each  Military  Department 
gathers,  tracks,  and  computes  operating  and  support  costs  by 
weapon  system.  VAMOSC  II  is  an  Air  Force  management  information 
system  which  is  responsive  to  the  OSD  initiative.  It  uses  infor¬ 
mation  from  existing  Air  Force  systems  to  satisfy  both  Air  Force 
and  OSD  needs  for  certain  weapon  system  operating  and  support  (O&S) 
costs. 

At  present,  the  VAMOSC  II  system  comprises  three  subsystems: 

(1)  The  Weapon  System  Support  Cost  (WSSC)  system  (D160) , 
which  deals  with  aircraft, 

(2)  The  Communications  -  Electronics  (C-E)  system  (D160A) , 
which  deals  with  ground  communications  -  electronics 
equipment, 

(3)  The  Component  Support  Cost  Subsystem  (CSCS)  (D160B) , 
which  deals  with  subsystems  and  components  for  aircraft. 

The  Component  Support  Cost  System  (CSCS)  of  VAMOSC  II  gathers 
and  computes  support  costs  by  assembly/subassembly  and  relates 
those  costs  back  to  the  end  item  or  weapon  system.  CSCS  replaces 
the  Logistic  Support  Cost  (LSC)  model  of  K051  (AFLCR  400-49)  for 
aircraft  and  engines.  - 

The  objectives  of  the  Component  Support  Cost  System  are: 

(1)  To  improve  the  visibility  of  aircraft  and  engine  com¬ 
ponent  support  costs  and  to  relate  those  costs  to  the 
end  item  or  weapon  system. 
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(2)  To  improve  the  Life  Cycle  Costing  capability  for  the 
Air  Force  and  the  Department  of  Defense  in  the  acqui¬ 


sition  of  new  weapon  systems. 

(3)  To  assist  in  the  design  of  new  weapon  systems  by  pro¬ 
viding  cost  information  on  existing  weapon  systems, 
thereby  enhancing  design  tradeoff  studies. 

(4)  To  provide  historical  cost  information  at  the  weapon 
system  level  to  improve  logistic  policy  decisions. 

(5)  To  identify  system  component  reliability,  effective¬ 
ness,  and  costs  so  that  high  support  cost  items  may 
be  identified  and  addressed. 

The  CSCS  receives  inputs  from  15  Air  Force  data  systems.  On 
a  quarterly  basis,  the  system  provides  two  standard  reports  each 
processing  cycle  and  twelve  other  types  of  reports  as  requested 
by  users.  It  also  provides  pre-programmed  data  base  extracts  on 
magnetic  tape  on  a  one-time  basis  in  response  to  user  requests. 

Special  requests  for  data  in  user  selected  format  may  also  be 
satisfied  on  a  case  by  case  basis. 

At  the  heart  of  the  CSCS  is  a  set  of  30  algorithms  for  esti¬ 
mation  or  allocation  of  costs.  Table  1  identifies  the  algorithms 
by  name.  Information  Spectrum,  Inc.  (ISI)  was  awarded  a  contract 
to  validate  these  algorithms.  This  effort  included  investigations 
of  logic,  appropriateness  of  the  algorithms,  and  assumptions  inherent 
in  the  algorithms.  ISI  was  also  to  survey  published  findings,  reports 
of  audit,  etc.  relating  to  the  accuracy  of  the  source  data  systems. 

The  analyses  of  the  CSCS  algorithms  were  called  "general  tasks" 
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TABLE  I-  CSCS  ALGORITHM  NAMES 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 
9. 

10. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 
21. 
22. 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 


Base  TCTO  Labor  Cost 

Base  TCTO  Overhead  Cost 

Base  TCTO  Material  Cost 

TCTO  Transportation  Costs 

Base  Inspection  Costs 

Base  Other  Support  General  Costs 

Base  Labor  Costs 

Base  Direct  Material  Costs 

Base  Maintenance  Overhead  Costs 

Second  Destination  Transportation  Costs 

Second  Destination  Transportation  Costs  (Engine) 

Base  Exchangeable  Repair  Costs  (NSN) 

Base  Exchangeable  Repair  Costs  (Engine) 

Base  Exchangeable  Modification  Costs  (NSN) 

Base  Condemnation  Spares  Costs/NSN 

Base  Exchangeable  Modification  Costs  (Engine) 

Base  Supply  Management  Overhead  Costs 

Depot  TCTO  Labor  Costs 

Depot  TCTO  Material  Costs 

Depot  TCTO  Other  Costs 

Depot  Support  General  Costa 

Depot  Labor  Costs 

Depot  Direct  Material  Costs 

Depot  Other  Costs 

Depot  Exchangeable  Repair  Costs  (NSN) 

Depot  Exchangeable  Repair  Costs  (Engine) 

Depot  Exchangeable  Modification  Costs  (NSN) 

Depot  Exchangeable  Modification  Costs  (Engine) 
Depot  Condemnation  Spares  Costs  (NSN) 

Depot  Material  Management  Overhead  Cost 
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by  the  contract.  In  addition,  two  "specific  tasks”  were  assigned. 

One  of  these  tasks  concerned  the  "NSN/WUC  Cross  Reference  Dictionary." 
Base  maintenance  costs  and  the  bulk  of  CSCS  outputs  are  expressed 
in  terms  of  Work  Unit  Codes  (WUCs) .  Base  and  depot  supply  infor¬ 
mation  and  depot  maintenance  costs  are  collected  in  terms  of  item 
National  Stock  Numbers  (NSNs).  One  of  the  more  critical  parts  of 
the  CSCS  logic  was  the  development  and  maintenance  of  a  NSN/WUC 
Cross  Reference  Dictionary.  ISI's  first  specific  task  was  to  devise 
a  suitable  methodology  to  verify  accuracy  of  the  dictionary,  as 
well  as  techniques  to  insure  continued  accuracy. 

One  of  the  CSCS  algorithms  allocates  base  supply  management 
overhead  costs  to  the  individual  WUCs.  The  Office  of  VAMOSC  (OOV) 
questioned  the  accuracy  of  the  methodology  used  in  this  algorithm. 

ISI's  second  specific  task  was  to  develop  a  new  method  for  allocating 
base  supply  management  overhead  costs  to  WUCs  on  a  quarterly  basis. 

In  its  response  to  the  government's  solicitation*15.  Information 
Spectrum  proposed  also  to  investigate  the  usefulness  of  CSCS  output 
reports  to  the  user  community.  Moreover,  in  the  earliest  stages  of 
the  effort  ISI  recognized  the  desirability  of  generating  a  final  report. 
It  summarizes  the  activities  of  the  project.  It  compiles  the 
recommendations  which  arose  out  of  the  analyses  of  the  individual 
algorithms.  Compilation  is  appropriate  because  some  recommendations 
are  common  to  several  algorithms. 


*15  Solicitation  number  F33600-82-R-0307 
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This  report  also  provides  a  vehicle  for  presentation  of  an 
analysis  of  the  survey  of  CSCS  users.  Finally,  the  project 
engendered  several  recommendations  which  relate  to  the  Component 
Support  Cost  System  as  a  whole,  rather  than  to  any  particular 
algorithm.  It  was  not  appropriate  to  include  these  recommendations 
in  any  of  the  reports  which  addressed  the  individual  algorithms; 
fchey  are  provided  here. 


2.0  SUMMARY  OF  PROJECT  ACTIVITIES 

Information  Spectrum,  Inc.  began  work  on  this  project  in 
mid  August,  1982,  and  completed  it  in  August,  1984.  During 
this  period  development,  modification,  and  validation  of  the  CSCS 
programs  went  on.  At  any  stage  of  its  effort,  ISI  accepted  current 
descriptions  of  the  program  functions  as  definitive. 

The  largest  part  of  the  activity  was  devoted  to  validation 
of  the  30  CSCS  cost  algorithms.  That  effort  is  described  in 
section  2.1  of  this  report.  The  "specific  task"  dealing  with  the 
NSN/WUC  Cross  Reference  Dictionary  was  also  a  significant  effort. 

The  activities  are  described  in  section  2.2.  The  other  "specific 
task,"  development  of  a  new  methodology  for  dealing  with  base  supply 
management  overhead  cost,  was  incorporated  into  the  analysis  of  the 
existing  algorithm  addressing  that  cost. 

The  survey  of  CSCS  users  took  place  toward  the  end  of  the 
project.  Section  2.3  summarizes  this  activity. 

2.1  Algorithm  Analyses 

Early  activities  included  development  of  an  algorithm  analysis 
methodology  and  a  report  format  applicable  to  analysis  of  all 
algorithms. 

The  CSCS  uses  input  data  from  15  other  Air  Force  data  systems. 

It  reflects  maintenance  and  supply  procedures  at  both  base  and 
depot  levels.  It  is  strongly  influenced  by  Department  of  Defense 
cost  analysis  policy  and  guidance.  For  all  these  reasons,  it  was 
appropriate  to  collect  and  review  a  large  number  of  reference  docu¬ 
ments.  By  the  end  of  the  project  more  than  120  regulations,  manuals, 
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reports,  etc.  had  been  catalogued.  Many  of  these  provided  signi¬ 
ficant  contributions  to  the  algorithm  analysis  process. 

In  some  cases  the  CSCS  algorithms  were  sufficiently  similar 
to  each  other  so  that  it  was  appropriate  to  discuss  two  o^  more 
algorithms  in  a  single  report.  In  total,  the  analyses  of  the  30 
algorithms  resulted  in  19  reports,  which  are  identified  in  Table  2. 

These  reports  have  been  provided  separately  to  the  Office 
of  VAMOSC.  The  OOV  desired  that  each  report  stand  alone  —  that 
is,  be  complete  without  the  need  for  reference  to  another  report. 
Accordingly,  some  material  on  the  background  of  the  CSCS  and  on 
the  general  methodology  is  repeated  in  all  of  the  reports.  Also, 
it  proved  convenient  to  provide  a  single  list  of  references  common 
to  all  reports,  although  any  single  report  uses  only  a  portion  of 
the  references. 

The  solicitation  identified  aspects  of  the  algorithms  to  be 
considered.  These  aspects  concerned  accuracy,  assumptions,  directness 
of  costing,  etc.  For  each  algorithm,  each  of  these  aspects  was  con¬ 
sidered.  Each  was  either  affirmed  or  rejected  by  ISI.  Whenever 
some  aspect  of  an  algorithm  was  rejected,  ISI  provided  a  suitable 
recommendation.  Drafts  of  the  analyses,  including  these  recommenda¬ 
tions,  were  provided  to  the  Office  of  VAMOSC.  The  OOV  provided  a 
written  response  to  each  recommendation.  In  general,  these  replies 
identify  the  actions  which  the  OOV  plans  to  take  in  response  to 
the  recommendations.  Tw^y  are  incorporated  into  the  algorithm  analysis 
reports.  ISI's  recommendations  regarding  the  CSCS  cost  algorithms 
are  compiled  in  Appendix  A  of  this  report,  and  summarized  in  Section 
3.1.1. 
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Table  2.  CSCS  Algorithm  Analysis  Reports 


tfo. 


(1) 


Subject(2) 


Algorithms 

Addressed 


01 

02 

03 

04 

05 

06 

07 

08 

09 


L0 


11 

12 

13 

14 

15 

16 


17 

18 
19 


Base  TCTO  Labor  Costs 

Base  TCTO  Overhead  Costs 

Base  TCTO  Material  Costs 

Base  Inspection  Costs 

Base  Other  Support  General  Costs 

Base  Labor  Costs 

Base  Direct  Material  Costs 

Base  Maintenance  Overhead  Costs 

Base  Exchangeable  Repair  Costs  (MSN)  and 
Base  Exchangeable  Modification  Costs  (NSN) 

Base  Exchangeable  Repair  Costs  (Engine)  and 
Base  Exchangeable  Modification  Costs  (Engine) 

Base  and  Depot  Condemnation  Spares  Costs  (NSN) 

Depot  TCTO  Labor  Costs 

Depot  TCTO  Material  Costs 

Depot  TCTO  Other  Costs 

Depot  Support  General,  Labor,  Direct 
Material,  and  Other  Costs 

Depot  Exchangeable  Repair  and  Modification 
Costs  for  NSNs  and  Engines 

Base  Supply  Management  Overhead  Costs 

Depot  Material  Management  Overhead  Costs 

Second  Destination  Transportation  Costs 


1 

2 

3 

5 

6 

7 

8 
9 

12,14 

13,16 

15,29 

18 

19 

20 

21,22,23,24 

25,26,27,28 

17 

30 

4,10,11 


(1)  Information  Spectrum  report  numbers  for  this  series  are  of  the 
form  V-83-31859-nn,  where  nn  is  the  number  in  this  column. 


(2) 

Each  CSCS  algorithm  analysis  report  title  is  of  the  form 
"Validation  of  the  Algorithm  for  (Subject)  for  the  Component 
Support  Cost  System  (D160B) 


(3) 


As  numbered  in  Table  1. 


2.2  NSN/WUC  Cross  Reference  Dictionary  Activities 
The  consideration  of  the  NSN/WUC  Cross  Reference  Dictionary 

involved  analysis  of  complex  paths  of  logic  and  procedure.  A 
variety  of  references  were  consulted?  eleven  are  cited  in  the 
report.  Frequent  contact  with  Air  Force  personnel  in  person  and 
by  telephone  led  to  an  understanding  of  the  established  procedures 
for  generating  the  dictionary,  updating  it,  and  using  it.  Careful 
consideration  of  the  information  led  to  questions,  which  were 
resolved  by  further  oral,  and  sometimes  written,  contacts. 

As  a  detailed  understanding  of  the  dictionary  developed,  the 
analyst  focused  his  attention  on  possible  sources  of  inaccuracy 
in  its  development,  maintenance,  and  application.  Development  of 
appropriate  recommendations  was  a  creative  process  which  was 
reinforced  through  further  discussion  with  cognizant  Air  Force 
personnel.  The  results  were  assembled  into  a  report  which  was 
presented  in  draft  form  to  the  Office  of  VAMOSC  for  their  critique. 
After  ISI  resolved  any  uncertainties  thus  revealed,  a  final  version 
was  issued. 

2. 3  Base  Supply  Management  Overhead  Costs 
Analysis  and  critique  of  the  algorithm  currently  in  the  CSCS 

for  allocating  base  supply  management  overhead  costs  was  a  natural 
part  of  the  "general  task"  required  by  the  solicitation.  ISI,  like 
the  Office  of  VAMOSC,  considered  this  existing  algorithm  unsatisfactory. 
Accordingly,  recommendation  of  an  improvement  was  also  called  for 
by  the  general  task,  and  was  achieved  just  as  were  other  recommenda¬ 


tions.  Thus,  although  the  development  of  a  modified  method  for 
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allocating  base  supply  management  overhead  costs  was  identified 
as  a  specific  task,  the  activities  were  part  of  those  used  for 
the  algorithm  analysis  performed  as  a  general  task. 


2. 4  User  Survey 

As  mentioned  earlier,  the  CSCS  continued  to  evolve  throughout 
the  period  of  this  effort.  Accordingly,  the  user  survey  was 
delayed  as  long  as  possible.  ISI  developed  a  questionnaire,  which 
is  reproduced  here  as  Figure  1.  That  questionnaire  was  distributed 
by  the  00V  on  March  20,  1984  to  141  addressees  in  106  organizations 
The  addressees  comprised  all  (56)  continuing  recipients  of  CSCS 
reports,  all  who  have  ever  received  a  one-time  report,  and  all  on 
the  mailing  list  for  the  annual  tri-service  VAMOSC  conferences. 

Thirty-three  replies  were  received  by  the  OOV,  and  copies  were 
provided  to  ISI.  These  were  analyzed  as  part  of  the  preparation 
of  this  final  report,  and  as  described  in  Appendix  B.  Responses 
were  tabulated  and  subjected  to  qualitative  analysis.  The  results 
of  this  tabulation  and  analysis  also  appear  in  Appendix  B. 


PLEASE  COMPLETE  THIS  QUESTIONNAIRE 
FOLD,  STAPLE  AND  MAIL  TO  THE  ADDRESS 
ON  THE  REVERSE  BY  1  MAY  1984 


Are  you  familiar  with  the  CSCS  Users  Manual.  Circle  one  (Y)  (N) 

a.  If  yes,  does  it  provide  an  adequate  description  of  the  CSCS 
data  available?  Circle  one  (Y)  (N) 

If  not  familiar  with  CSCS  data  products,  what  needs  do  you  have 
for  historical  operating  and  support  (OSS)  costs  at  the  aircraft 
component  (WUC)  level?  _ 


3.  List  reports/data  you  currently  receive  from  the  CSCS.  Identify  data 
elements  that  are  of  particular  interest  to  you  by  their  titles 
on  the  report.  _ 


4.  Do  the  reports  you  currently  receive  provide  all  the  information  you 

require?  Circle  one.  (Y)  (N) 

5.  Indicate  information  you  would  like  to  see  from  CSCS  that  is  not 

included  in  current  reports.  _ 


6.  Do  you  consider  the  CSCS  data  to  be  useful?  Circle  one.  (Y)  (N) 

7.  Is  the  format  of  the  current  output  reports  satisfactory.  Circle  one. 

(Y)  (N) 

a.  Is  the  media  of  the  current  output  reports  (microfiche,  tape) 
satisfactory?  Circle  one.  (Y)  (N) 

8.  How  in  your  opinion  can  the  CSCS  output  reports  be  improved?  _ 


9.  Do  you  have  a  continuing/regular  need  for  CSCS  data.  Circle  one.  (Y)  (N) 

10.  Is  the  meaning  of  the  information  on  CSCS  reports  clear  and  understood? 
Circle  one.  (Y)  (N) 


a.  Explain  if  NO. 


In  your  opinion,  would  a  training  package  in  the  derivation/definition 
of  CSCS  data  be  useful?  Circle  one  (Y)  (N) 

Are  the  CSCS  quarterly  output  reports  timely  enough  for  your  purposes? 
Circle  one.  (Y)  (N) 


Indicate  the  functional  areas  in  which  you  have  an  interest  in 
aircraft  component  costs. 


(  )  Logistics 

(  )  Budget  Preparation 

(  )  Planning  and  Programming 


(  )  Cost  Estimating 
(  )  Procurement  (Acquisition) 
(  )  Other  (Specify) 


14.  What  do  you/have  you  used(d)  the  CSCS  data  for? 


15.  Provide  other  comments  as  desired. 


(PLEASE  PROVIDE  RETURN  ADDRESS  ON  REVERSE) 


3.0  RESULTS 


The  study  led  to  many  recommendations,  which  are  discussed 
in  Section  3.1.  The  validation  of  the  accuracy  of  source  data 
is  an  area  of  special  interest,  and  is  addressed  in  Section  3.2. 
Results  of  the  user  survey  are  reviewed  in  Section  3. 3. 

Each  CSCS  output  report  contains  cost  information  generated 
by  several  algorithms.  Section  3.4  considers  the  accuracy  of  the 
reports. 

3. 1  Recommendations 

Section  3.1.1  discusses  recommendations  arising  out  of  the 
algorithm  analysis.  Section  3.1.2  addresses  those  which  resulted 
from  the  investigation  of  the  NSN/WUC  Cross  Reference  Dictionary. 

3.1.1  Algorithms 

In  many  cases,  several  CSCS  cost  algorithms  share  common 
techniques.  Therefore,  a  recommendation  may  apply  to  several 
algorithms,  and  may  be  presented  in  more  than  one  of  the  reports  of 
Table  1.  As  mentioned  in  Section  2.1  of  this  report,  repetition 
occurs  because  each  of  the  algorithm  analysis  reports  was  to  be 
independent  of  the  others. 

For  this  Final  Report,  a  single  consolidated  list  of  recommenda 
tions  was  prepared.  It  is  presented  in  Appendix  A,  and  it  provides 
18  recommendations.  In  each  case,  the  problem  leading  to  the 
recommendation  is  summarized  first,  under  the  heading  "Criticism." 
Then  the  recommendation  is  listed. 


The  Office  of  VAMOSC,  in  order  to  assure  the  effectiveness  and 


usefulness  of  this  effort,  provided  a  written  response  to  each 
recommendation.  In  almost  every  case,  these  responses  identified 
the  action  which  would  be  taken.  Those  responses  are  included  in 
the  individual  algorithm  analysis  reports,  and  are  summarized  under 
the  headings  "Response”  in  Appendix  A. 

The  criticisms,  recommendations,  and  responses  presented  in 
Appendix  A  are  digests.  For  a  full  understanding  of  the  criticisms 
and  recommendations,  and  the  full  text  of  the  00V  responses,  it  is 
imperative  that  the  algorithm  analysis  reports  be  consulted. 

Section  2.1  mentioned  that  each  aspect  (as  identified  by  the 
solicitation)  of  each  algorithm  was  either  affirmed  or  rejected  by 
IS I.  Each  recommendation  corresponds  to  a  rejection  of  some  aspect 
(sometimes  more  than  one)  of  an  algorithm.  As  mentioned,  some 
recommendations  apply  to  several  algorithms.  Six  of  the  30  algorithms 
were  fully  affirmed,  with  retention  in  their  present  form  recommended. 
These  are  the  algorithms  called  Base  TCTO  Material  Costs,  Base  Direct 
Material  Costs,  Depot  TCTO  Labor  Costs,  Depot  TCTO  Material  Costs, 
Depot  TCTO  Other  Costs,  and  Depot  Material  Management  Overhead  Costs. 

3.1.2  Cross-Reference  Dictionary 

As  indicated  in  Section  2.2,  the  study  of  the  NSN/WUC  Cross- 
Reference  dictionary  required  analysis  of  complex  paths  of  logic 
and  procedure.  The  constructive  result  of  this  effort  was  a  list 
of  recommendations  oriented  toward  correction  or  avoidance  of 
inaccuracies  in  the  development,  maintenance,  and  application  of 
the  Dictionary.  The  analysis  and  recommendations  are  presented 


in  ISI  Report  No.  V-84-31859-20,  "An  Investigative  Analysis  of 
the  NSN-WUC  Cross-Reference  Dictionary  for  the  Component  Support 
Cost  System  (D160B),"  21  September  1984.  Section  8  of  that  report 
lists  30  recommendations.  Even  for  the  most  knowledgeable  reader, 
it  would  be  necessary  to  study  the  report  carefully  in  order  to 
understand  the  import  of  the  recommendations. 

The  recommendations  are  organized  by  areas  of  analysis.  The 
first  area,  "File  Integrity,"  is  concerned  with  the  validity  and 
accuracy  of  the  Dictionary.  The  report  provides  11  recommendations. 
"Workload  Management"  is  concerned  with  efficient  procedures  for 
Dictionary  maintenance;  7  recommendations  are  presented.  The 
relationships  among  Work  Unit  Codes  and  stock  numbered  items, 
especially  in  view  of  interchangeability  and  substitutability, 
include  subtle  problems.  These  are  addressed  by  7  recommendations 
titled  "Cost  Output  Usability."  The  complex  relationships  among 
Dictionary  elements  suggest  some  modernization  in  the  approach  to 
file  structure.  Three  recommendations  are  provided  under  the  heading 
"Data  Base  Organization."  Finally,  two  recommendations  address  the 
special  problems  associated  with  their  title,  "Introduction  of  a 
New  MDS." 

3.2  Input  Data  System  Accuracy 

As  part  of  its  algorithm  analysis,  ISI  was  required  to  validate 
the  accuracy  of  source  data  systems  providing  inputs  to  the  CSCS. 

This  task  was  to  include  survey  of  published  findings,  reports  of 
audit,  etc.  Sampling  of  data  was  not  to  be  performed.  This  topic 
is  given  special  attention  in  this  report  because  of  widespread 
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interest  in  a  report  known  in  Air  Force  logistic  circles  as  "the 
GAO  report. This  report  assails  the  data  in  the  Maintenance 
Data  Collection  System  (MDCS)  for  inaccuracy  and  lack  of  timeliness. 

The  GAO  report  often  relies  on  small  samples,  and  it  is  more  anecdotal 
than  scientific.  Nevertheless,  as  a  whole  it  is  convincing. 

Information  Spectrum  also  identified  a  study  whose  results 
are  incorporated  (but  not  explicitly  identified)  in  the  GAO  report. 
Although  that  study  could  not  be  freely  extrapolated  to  all  main¬ 
tenance  events,  it  strongly  suggested  that  many  maintenance  events 
are  not  reported  through  the  MDCS,  while  man-hours  are  exaggerated 
for  those  that  are  reported. 

The  Air  Force  is  testing  an  automated  system  which  holds  promise 
of  considerably  improving  the  accuracy  of  reporting  of  maintenance 
manhours.  This  system,  called  the  Core  Automated  Maintenance  System 
(CAMS) ,  provides  for  real  time,  automated  input,  editing,  and  retrieval 
of  data  of  the  MDCS.  The  CAMS  is  currently  being  tested  at  Langley 
AFB.  The  GAO  report  does  not  provide  direct  evidence  of  improved 
accuracy  provided  by  the  CAMS,  but  it  cites  impressive  improvements 
in  the  number  of  maintenance  actions  reported  as  completed.  It  also 
indicates  that  Air  Force  officials  believe  that  the  CAMS  virtually 
eliminates  inaccuracy  in  MDC  data. 

(1)  Report  by  the  Comptroller  General  of  the  United  States  of  the 
Chairman,  Committee  on  Government  Operations,  House  of 
Representatives,  "The  Air  Force  Can  Improve  Its  Maintenance 
Information  Systems,"  GAO  Report  No.  GGD-83-20,  January  25,  1983. 


ISI  could  find  no  alternative  to  using  MDCS  data  in  the 
immediate  future,  despite  any  accuracies  which  it  may  introduce 


into  base  labor  cost  estimates.  The  CAMS,  when  introduced,  should 
provide  considerable  improvement.  No  change  in  format  of  data 
provided  to  the  CSCS  is  expected,  so  no  changes  to  that  system  will 
be  needed. 

For  all  of  the  other  data  systems  which  provide  input  to  the 
CSCS,  no  published  criticisms  were  found. 

3. 3  User  Survey 

The  results  of  the  user  survey  are  incorporated  in  this  report 
as  Appendix  B.  Recommendations  included  there  are  summarized  as 
follows: 

•  Survey  CSCS  users  periodically.  Review  responses,  con¬ 
tact  responders  to  clarify  their  answers,  and  establish 
procedures  for  reacting  to  responses. 

•  Study  the  individual  responses  to  those  questions  of 
the  questionnaire  which  called  for  user  comments. 

•  Develop  a  training  package  about  the  derivation  and 
definition  of  CSCS  data. 

3.4  CSCS  Output  Report  Accuracy 
Each  of  the  individual  algorithm  analysis  reports  noted  that 

the  total  accuracy  of  the  CSCS  output  reports  could  not  be  con¬ 
sidered  until  all  algorithms  were  reviewed.  Accordingly  this 
accuracy  is  addressed  here. 


ISI  considered  extracting  from  each  report  a  list  of  output 
data  elements,  and  on  the  basis  of  the  algorithm  analysis  reports 
branding  each  element  as  accurate  or  inaccurate.  A  first  attempt 
convinced  us  that  this  approach'  would  not  provide  satisfactory 
insight.  The  following  list  of  findings  relevant  to  output  report 
accuracy  is  appropriate. 

(1)  There  were  3Q  cost  algorithms  considered.  ISI  recommended 
changes  in  24,  and  no  change  in  the  other  6. 

(2)  Most  of  the  changes  should  provide  only  small  improve¬ 
ments  in  the  accuracy  of  calculated  costs.  For  example,  one 
change  of  wide  applicability  is  the  institution  of  a  procedure 

for  recalculating  base  maintenance  labor  rates,  rather  than  inflating 
the  rates  established  in  1980.  Another  is  the  use  of  annual,  rather 
than  quarterly,  averages  of  various  kinds  of  depot  activity  rates. 

(3)  Many  of  the  cost  elements  appearing  on  output  reports 
include  the  effects  of  several  algorithms.  For  example,  total 
base  and  depot  Work  Unit  Code  costs  include  the  effects  of  all  of 
the  algorithms.  Again,  the  Logistic  Support  Cost  Ranking  report 
implicitly  includes  the  effect  of  all  of  the  algorithms  in  the 
ranking. 

(4)  The  meanings  of  some  of  the  cost  elements  on  the  reports 
are  not  clear  from  their  titles  or  from  any  available  documentation. 
00V  personnel  have  expressed  a  need  to  have  programming  personnel 
provide  precise  definitions. 

(5)  The  reports  include  some  numeric  items  which  are  not 
costs.  For  instance,  some  display  owned  inventory,  flight  hours, 


sorties,  etc.  Consideration  of  these  was  outside  the  scope  of 
this  effort. 

(6)  In  the  course  of  this  effort,  it  became  evident  that 
any  inaccuracies  in  the  output  of  the  algorithms  were  being  ob¬ 
scured  by  data  processing  problems.  Investigation  of  these  was  out 
side  the  scope  of  this  effort.  Problems  may  have  been  due  to  bad 
input  data  or  to  programming  errors.  Problems  noted  included  the 
following: 

(a)  Large  quantities  of  data  are  being  reported  against 
Work  Unit  Codes  or  stock  numbers  which  could  not 

be  adequately  identified.  This  could  be  due  to 
input  equipment  identification  errors  or  to  the 
incomplete  state  of  the  Cross-Reference  Dictionary. 

(b)  Many  output  quantities  are  incorrectly  displayed 
as  zeros.  Note  that  missing  data  should  not  be 
reported  as  zero. 

(c)  Some  outputs  are  not  credible.  For  instance,  the 
latest  CSCS  output  shows  base  maintenance  overhead 
rates  for  the  F-16A  from  $0.94  (Kadena)  to  $76.78 
(Zaragoza) .  Such  a  range  is  prima  facie  not 
credible. 

In  summary,  ISI  believes  that  the  majority  of  the  cost  data 
on  CSCS  output  reports  would  have  some  deficiencies  based  on  our 
criticisms,  if  they  accurately  reflected  the  current  design  of 
the  algorithms.  However,  these  deficiencies  are  currently  obscured 
by  other  data  processing  problems.  Information  Spectrum  recommends 
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that  the  Office  of  VAMOSC  support  a  validation  effort  which  would 
trace  numerical  data  through  the  CSCS,  validating  intermediate 
results.  Such  an  effort  should  locate  and  correct  errors,  and 
provide  clarified  program  documentation.  It  would  result  in 
a  thorough  internal  program  validation. 


APPENDIX  A 


COMPILATION  OF  CRITICISMS,  RECOMMENDATIONS, 

AND  RESPONSES  FOR  CSCS  COST  ALGORITHMS 

Many  of  the  CSCS  cost  algorithms  share  common  elements.  For 
instance,  the  algorithms  titled  Base  TCTO  Labor  Cost,  Base  Inspec¬ 
tion  Costs,  Base  Other  Support  General  Costs,  and  Base  Labor  Cost 
all  use  the  same  average  base  labor  rates.  Therefore,  the  reports 
which  address  the  algorithms  sometimes  contain  identical  recom¬ 
mendations. 

This  appendix  compiles  the  recommendations  of  the  analysis 
reports. ^  For  each  topic,  the  appendix  identifies  the  criticism, 
the  recommendation,  and  the  Office  of  VAMOSC  (00V)  response.  None 
of  these  is  addressed  in  full  detail  in  this  appendix.  For  a  full 
understanding  of  the  criticisms  and  recommendations,  and  the  full 
text  of  the  OOV  responses,  it  is  imperative  that  the  algorithm 
analysis  reports  be  consulted.  These  reports  are  identified  in 
Table  1  of  the  body  of  this  report. 

1.  Base  Direct  Labor  Rates 

Criticism;  Several  algorithms  use  the  average  military  base  level 
direct  labor  rate  for  each  MDS.  These  rates  were  calculated  based 
on  a  1980  sample,  and  are  being  inflated  for  later  years.  This 
will  lose  validity  as  time  elapses.  Moreover,  this  single  military 
rate  is  applied  to  both  military  and  civilian  man-hours. 

(2) 

Recommendation ;  Report  No.  06  recommends  an  automated  methodology 

It  does  not  include  the  recommendations  for  the  Cross-Reference 

Dictionary.  See  Section  3.1.2. 

See  Table  1. 


to  calculate  both  military  and  civilian  base  level  direct  labor 
rates  for  each  MDS  every  quarter.  The  procedure  is  similar  to 
that  which  was  used  for  the  1980  data.  No  escalation  for  infla¬ 
tion  is  needed. 

Response :  The  00V  concurs.  It  will  review  the  proposal  prior  to 
FY  86,  when  changes  to  the  Maintenance  Data  Collection  System  are 
expected  to  become  possible.  In  the  interim,  the  OOV  will  use 
rates  calculated  manually,  based  on  reports  from  the  Maintenance 
Cost  System,  representing  a  weighted  mix  of  military  and  civilian 
hours . 

2.  Military  and  Civilian  Labor  at  Base  Level 

Criticism:  At  present,  military  and  civilian  base  level  mainte¬ 
nance  man-hours  are  added,  and  multiplied  by  a  military  direct  labor 
rate.  This  is  inaccurate. 

Recommendation :  ISI  recommends  that  military  and  civilian  man¬ 
hours  be  provided  to  the  CSCS  separately.  These  should  have 
separate  labor  rates  applied.  Civilian  and  military  labor  costs 
should  be  displayed  separately. 

Response:  OOV  found  from  a  sample  that  only  two  to  three  percent 
of  the  total  base  maintenance  work  force  are  civilians.  There¬ 
fore,  the  utility  of  separate  costing  of  military  and  civilian 
labor  is  questionable.  (The  use  of  a  weighted  composite  rate 
instead  of  a  pure  military  rate,  identified  in  the  response  to 
item  1  above,  provides  a  simpler,  if  less  precise,  solution.) 
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00V  will  further  review  this  proposal.  They  question  the  utility 
of  displaying  civilian  and  military  labor  costs  separately  on 
output  reports. 


3.  Inflation  Factors 

Criticism:  In  escalating  labor  rates,  the  CSCS  uses  annual  infla¬ 

tion  indexes  published  by  the  Department  of  Defense.  The  quarterly 
CSCS  reports  will  be  based  on  labor  rates  showing  no  change  for 
three  quarters,  and  a  full  year's  worth  of  change  the  fourth. 

This  needlessly  introduces  irregularity  into  calculated  labor 
costs.  (However,  it  now  appears  that  the  00V  will  calculate  new 
labor  rates  as  pay  rates  change,  which  will  solve  the  problem.) 

Recommendation ;  ISI  recommends  simple  linear  interpolation  formulas 
which  provide  for  applying  an  appropriate  portion  of  inflation 
each  quarter. 

Response:  The  00V  concurs. 

4.  System  Documentation 

Criticism:  Three  documents  were  primary  references  in  ISI's  anal¬ 
yses.  They  are  known  as  the  User's  Manual,  the  Functional  Descrip¬ 
tion,  and  the  System  Specification.  These  documents  are  also  primary 
vehicles  for  communication  within  the  00V.  They  were  found  to  be 
significantly  deficient  in  accuracy,  clarity,  completeness,  and 
consistency.  We  believe  that  this  will  hamper  future  development 
of  the  CSCS. 


Recommendation :  ISI  recommends  that  the  three  documents  be  revised. 


We  recommend  a  significant  allocation  of  attention,  time,  and 
resources . 

Response:  The  00V  indicates  that  revisions  and  updates  are  in  pro 
cess  for  all  documents.  Until  an  acceptable  level  of  consistency 
is  obtained  and  a  system  to  maintain  that  level  is  implemented, 
this  will  remain  a  major  work  effort.  The  00V  expected  to  be  in 
a  satisfactory  posture  regarding  this  area  by  1  July  1984. 

5.  Input  Data  System  Error 

Criticism:  As  a  result  of  ISI  investigations,  personnel  of  the 
Air  Force  System  Design  Center  determined  that  the  D002A  system 
is  incorrectly  counting  local  manufacture  receipts  as  issues  in 
information  provided  to  the  CSCS. 

Recommendations :  Information  Spectrum  recommends  that  the  e.ro* 
be  corrected. 

Response :  The  00V  concurs. 

6.  Labor  Rates  for  Overhead  Costs 

Criticism:  As  a  part  of  the  calculation  of  base  maintenance  over¬ 
head  cost,  the  CSCS  calculates  base  maintenance  direct  labor  cost 
by  multiplying  man-hours  by  a  single,  Air  Force-wide,  direct  labor 
rate.  This  procedure  sacrifices  some  accuracy  which  is  available. 

^A  first  revision  to  the  System  Specification  was  received  after 
this  recommendation  was  made.  We  feel  that  further  revision  is 
still  needed. 


For  the  bulk  of  base  maintenance  labor,  the  man-hours  identified 
with  each  MDS  could  be  multiplied  by  the  labor  rate  applicable  to 
that  MDS. 


Recommendation :  It  is  recommended  that  the  procedure  for  develop¬ 

ing  the  Base  Overhead  Cost  Rate  be  refined  by  multiplying  the  DLR 
for  each  MDS  times  the  direct  labor  hours  that  can  be  identified 
to  that  MDS.  The  worldwide  single  DLR  would  be  applied  only  to 
all  those  base  direct  labor  hours  that  cannot  be  identified  to  an 
MDS,  such  as  labor  hours  for  GSE.  These  costs  would  be  added  to 
produce  a  more  accurate  Base  Direct  Labor  Cost. 

Response:  The  OOV  concurs.  A  suspense  date  for  DAR  submission  is 
31  January  1984,  and  projected  implementation  is  FY  87. 

7.  Depot  Activity  Rates 

Criticism:  Of  the  exchangeable  items  turned  in  to  a  depot,  some 
are  eventually  repaired,  some  are  modified,  and  some  are  condemned 
Several  algorithms  involve  estimates  of  these  proportions.  The 
estimates  for  each  WUC  are  based  on  the  experience  or  the  most 
recent  quarter. 

It  is  common  practice  to  accumulate  items  at  a  depot  for  many 
months,  and  then  to  process  them.  As  a  result,  estimates  of  pro¬ 
portions  based  on  one  quarter  of  data  are  not  expected  to  be 
representative . 

Recommendation :  ISI  recommends  that  estimates  of  depot  activity 

rates  be  based  on  the  most  recent  four  quarters  of  data.  Moreover 


if  no  items  of  a  particular  WUC  have  been  worked  on  for  the  latest 
four  quarters,  the  previous  estimates  of  depot  activity  rates 
should  be  retained. 


Response:  The  00V  concurs.  The  suspense  date  for  a  DAR  requesting 
this  change  is  31  May  1984. 

8.  Depot  Cost  Extrapolation 

Criticism:  The  current  method  used  to  compute  average  depot  repair 
and  modification  costs  relies  on  the  assumption  that  both  repair 
and  modification  take  place  for  every  NSN  in  every  quarter.  In 
the  event  that  no  such  activity  takes  place  for  a  particular  NSN 
in  a  particular  quarter,  the  program  incorrectly  uses  zero. 

Recommendation :  ISI  recommends  that  when  no  depot  repair  or  modi¬ 

fication  takes  place  for  a  particular  NSN  in  a  particular  quarter, 
the  CSCS  should  estimate  the  average  cost  by  using  the  figure  from 
the  previous  quarter,  adjusted  for  inflation. 

Response :  The  00V  concurs.  The  suspense  date  for  a  DAR  requesting 
this  change  is  31  May  1984. 

9.  Depot  MDS  Labor  Rates 

Criticism:  Several  algorithms  estimate  depot  labor  costs  by  multi¬ 
plying  depot  labor  hours  for  a  particular  kind  of  work  (Support 
General,  on-equipment  repair,  or  non-modification  TCTO)  by  an 
average  depot  labor  rate.  This  labor  rate  is  the  result  of 
dividing  total  depot  labor  costs  for  these  kinds  of  work  by  the 
corresponding  total  depot  labor  man-hours.  However,  the  man-hours 


used  for  this  rate  are  extracted  from  a  different  data  system 
(H036B)  than  is  used  for  the  man-hours  which  are  multiplied  by 
the  rate.  If  in  fact  the  total  man-hours  from  H036B  differs  from 
the  sum  of  man-hours  (from  several  systems)  for  Support  General, 
on-equipment  repair,  and  non-modification  TCTO,  then  the  resulting 
cost  estimates  will  be  distorted. 

Recommendation :  IS I  recommends  that  the  calculation  of  average 
depot  labor  rate  be  based  on  division  by  the  sum  of  the  same  man¬ 
hours  for  which  costs  are  to  be  allocated,  namely  Support  General, 
on-equipment  repair,  and  non-modification  TCTO. 

Response :  The  OOV  concurs.  The  suspense  date  for  a  DAR  requesting 
this  change  is  31  July  1984. 

10.  Depot  MDS  "Other"  Cost  Rates 

Criticism:  Depot  costs  are  categorized  as  labor,  material.  Support 
General,  and  "other."  "Other"  costs  are  allocated  through  develop¬ 
ment  of  an  average  "other"  cost  per  maintenance  man-hour.  The 
man-hours  are  developed  and  applied  exactly  the  same  way  as  for 
labor  rates  (item  9  above) ,  and  the  same  criticism  applies. 

Recommendation :  ISI  recommends  that  the  calculation  of  average 
depot  "other"  cost  rates  be  based  on  division  by  the  sum  of  the 
same  mam-hours  for  which  costs  are  to  be  allocated,  namely  Support 
General,  on-equipment  repair,  and  non-modification  TCTO. 

Response:  The  OOV  concurs.  The  suspense  date  for  a  DAR  requesting 
this  change  is  31  July  1984. 


11.  Depot  MDS  Material  Rates 

Criticism:  Depot  direct  material  costs  are  based  on  material  rates 
which  are  developed  and  applied  in  exactly  the  same  way  as  labor 
rates  (item  9  above) ,  and  the  same  criticism  applies. 

Recommendation :  IS I  recommends  that  the  calculation  of  depot 
average  material  rate  be  based  on  division  by  the  total  of  man-hours 
used  to  allocate  material  costs. 

Response :  The  OOV  concurs.  The  suspense  date  for  a  DAR  requesting 
this  change  is  31  July  1984. 

12.  Depot  Support  General  Other  Costs 

Criticism:  The  current  algorithm  for  depot  Support  General  costs 
considers  direct  labor  only.  "Other"  costs,  such  as  overhead  and 
G&A,  are  usually  allocated  in  proportion  to  labor  hours,  but  are 
currently  omitted. 

Recommendation :  IS I  recommends  that  the  depot  MDS  "other"  cost 
rate  be  added  to  the  labor  rate,  to  provide  a  total  cost  rate  to 
be  applied  to  depot  Support  General  man-hours. 

Response :  The  OOV  concurs.  The  suspense  date  for  a  DAR  requesting 
this  change  is  31  July  1984. 

13 .  Base  Supply  Management  Overhead 

Criticism:  The  Office  of  VAMOSC  recognized  that  the  existing 
method  for  calculating  base  supply  management  overhead  was  question¬ 
able.  One  feature  of  the  method  uses  survey  values  of  supply 
transactions  per  maintenance  action.  IS I  found  the  survey  methods 
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unverifiable,  and  the  results  not  credible.  Moreover,  we  found 
the  methodology  needlessly  complex. 


Recommendation ;  The  D002A  system  currently  provides  the  CSCS  with 
a  count  of  selected  supply  transactions.  (Reference  material 
indicates  that  the  count  is  not  the  one  desired  by  the  CSCS.) 

It  is  recommended  that  the  selection  criteria  be  changed,  so  that 
two  counts  may  be  supplied:  a  count  of  transactions  for  aircraft 
and  a  count  of  total  base  supply  transactions.  The  ratio  of  these 
counts  provides  an  estimate  of  the  proportion  of  supply  management 
overhead  costs  attributable  to  aircraft.  This  fraction  should  be 
multiplied  by  the  total  base  supply  management  overhead  cost, 
yielding  an  estimate  of  the  aircraft  supply  management  overhead 
cost.  This  cost  should  be  allocated  to  individual  aircraft  Work 
Unit  Codes  in  proportion  to  the  base  maintenance  direct  labor  man¬ 
hours  spent  on  repair  of  these  items. 

Response :  The  OOV  concurs.  Implementation  will  be  deferred  because 
the  D002A  system  is  deferring  implementation  of  DARs  until  that 
system  has  effected  transition  to  the  Phase  IV  computer  system. 

14 .  Shipping  Costs 

Criticism:  Information  Spectrum  determined  that  the  shipping  cost 
calculations  used  in  the  CSCS  are  unquestionably  an  attempt  to 
apply  methods  defined  by  AFLCP  173-10,  AFLC  Cost  and  Planning 
Factors.  That  methodology  appears  satisfactory,  but  several 
significant  errors  were  made  in  documenting  its  application  to 
the  CSCS.  These  errors  invalidate  the  calculated  shipping  costs. 


Moreover,  the  CSCS  calculations  omit  packaging  costs,  which 
are  more  significant  than  the  transportation  part  of  shipping  costs. 


Recommendation ;  We  recommend  that  the  shipping  cost  calculation  of 
AFLCP  173-10  be  followed  exactly,  including  packaging  costs. 

Response :  The  OOV  concurs.  The  suspense  date  for  a  DAR  requesting 
this  change  is  31  August  1984. 

15.  Engine  Shipments 

Criticism:  The  current  CSCS  algorithm  counts  receipts  of  engines 
by  depots  for  major  overhaul.  Each  such  report  generates  a  two- 
way  shipping  cost  between  base  and  depot.  Shipments  between  base 
and  depot  for  reasons  other  than  major  overhaul  are  possible,  but 
are  not  counted  by  the  algorithm.  Also,  a  significant  amount  of 
shipment  of  engines  is  between  bases  and  "Queen  Bee"  (regional 
intermediate  level  engine  repair)  facilities.  Such  shipments  are 
not  counted  by  the  algorithm. 

Recommendation :  ISI  recommends  that  all  reports  of  engine  receipts 
by  depots  be  used  to  generate  two-way  costs  of  shipping  between 
base  and  depot.  Reports  of  receipts  of  engines  shipped  from  one 
base  to  another  can  be  identified  through  the  D042  data  system. 
Reports  of  receipts  of  serviceable  engines  should  be  associated 
with  the  receiving  base.  Other  reports  should  be  associated  with 
the  shipping  base.  These  reports  should  be  used  to  generate  one¬ 
way  shipment  costs. 
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AY. 
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Response:  The  OOV  concurs.  The  suspense  date  for  DARs  supporting 
this  change  is  28  September  1984. 


16.  Engine  Shipping  Costs 

Criticism:  The  CSCS  converts  counts  of  engine  shipments  to  depots 
using  the  same  average  shipping  costs  per  pound  as  for  any  other 
shipped  material.  Air  Force  engine  management  personnel  have 
indicated  that  engine  shipment  procedures  have  unique  features 
which  cause  the  shipping  costs  to  differ  significantly  from  those 
generated  by  this  approach.  Each  engine  (identified  by  TMS)  tends 
to  have  its  own  shipping  characteristics. 

Recommendation :  The  Office  of  VAMOSC  should  support  an  investi¬ 
gation  into  actual  engine  shipping  costs.  The  investigation 
%  should  develop,  for  each  engine  or  module,  average  shipping  costs 
to  depot  for  CONUS  and  for  overseas  bases.  For  engines  or  modules 
maintained  at  Queen  Bee  facilities,  average  base  to  Queen  Bee  shipping 
costs  should  also  be  developed,  both  for  CONUS  and  overseas  bases. 

The  average  labor  cost  of  preparing  engines  for  shipment  should  also 
be  determined. 

The  CSCS  should  store  the  resulting  data  in  a  table,  and  use  it 
for  estimating  engine  shipping  costs.  The  table  should  be  updated 
annually  by  application  of  inflation  factors,  and  revised  periodically 
by  reiterated  study  efforts. 


Response :  The  OOV  concurs.  Contact  with  transportation  personnel 
at  OC-ALC  and  SA-ALC  has  been  initiated. 
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Data  Processing  for  Depot  Material  Management  Overhead  Costs 


Criticism:  In  the  System  Specification,  ISI  found  several  anomalies 
in  the  descriptions  of  the  applicable  data  processing  procedures. 

As  a  result,  there  is  inadequate  confidence  that  the  CSCS  output 
reports  and  magnetic  tapes  incorporate  material  management  overhead 
cost  elements  as  desired. 

Recommendation :  The  00V  should  confer  with  the  CSCS  programing 
activity  to  clearly  establish  the  contributions  of  depot  material 
management  overhead  cost  elements  to  CSCS  outputs,  and  to  correct 
any  deficiencies. 

Response :  The  OOV  concurs.  The  DAR  requesting  investigation  into 
the  actual  calculation  of  MMOH  costs  will  be  submitted  by  28  Sep  84. 

18.  Depot  Material  Management  Overhead  Factor 

Criticism:  The  material  management  overhead  factor  used  by  the 
CSCS  was  developed  for  other  purposes.  It  has  not  yet  been  possible 
to  identify  the  organization  which  developed  the  factor,  to  determine 
the  procedures  used  to  develop  it,  or  to  verify  its  suitability  for 
CSCS  application. 

Recommendation :  ISI  recommends  that  the  OOV  make  a  vigorous  effort 
to  identify  the  organization  which  developed  the  factor.  Through 
it,  either  the  suitability  of  the  factor  should  be  established, 
or  a  more  suitable  factor  should  be  developed.  In  either  event, 
supporting  documentation  should  be  provided. 


Response :  The  OOV  concurs.  The  CSCS  OPR  will  contact  those  indi¬ 
viduals  who  develop  the  MMOH  rates  found  in  AFR  26-1  and  solicit  their 
help  in  determining  the  applicability  of  those  rates  to  CSCS.  In 
addition,  procedures  will  be  set  up  to  ensure  that  VAMOSC  personnel 
are  alerted  to  any  future  updates  of  the  rates. 
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SURVEY  ANALYSIS 

As  indicated  in  the  text,  141  survey  questionnaires  were 
sent  out.  Thirty-three  responses  were  received.  Of  these,  9 
were  essentially  "no  comment"  responses. 

ISI  examined  the  remaining  24  responses,  and  tabulated  answers 
to  each  question.  In  some  cases  we  judged  that  an  answer  listed 
under  one  question  belonged  under  another. 

Figure  B-l  tabulates  the  responses.  Counts  of  the  form  Y/N 
tabulate  YES/NO  responses.  For  questions  calling  for  a  written 
response,  the  number  of  responses  was  counted.  Only  question  11 
was  answered  on  all  24  questionnaires. 

The  Office  of  VAMOSC  has  stated  that  there  are  56  regularly 
scheduled  recipients  of  CSCS  products.  There  were  15  responses 
to  question  3;  we  assume  that  these  correspond  to  regular  users. 
While  some  of  the  responses  to  the  survey  seemed  perfunctory,  many 
showed  a  sincere  interest  in  the  CSCS.  Information  Spectrum 
believes  and  recommends  that  regular  users  of  the  CSCS  should  be 
surveyed  periodically.  The  questionnaire  used  was  an  experimental 
model,  and  consideration  of  the  responses  suggests  areas  where  it 
could  be  improved.  Also,  the  Office  of  VAMOSC  should  allocate 
resources  to  review  of  responses,  contact  with  responders  to 
clarify  their  answers,  and  procedures  for  reacting  to  responses. 


PLEASE  COMPLETE  THIS  QUESTIONNAIRE 
FOLD,  STAPLE  AND  MAIL  TO  THE  ADDRESS 
ON  THE  REVERSE  BY  1  MAY  1984 


Are  you  familiar  with  the  CSCS  Users  Manual.  Circle  one  (Y)  (N) 

a.  If  yes,  does  it  provide  an  adequate  description  of  the  CSCS 
data  available?  Circle  one  (Y)  (N) 

If  not  familiar  with  CSCS  data  products,  what  needs  do  you  have 
for  historical  operating  and  support  (0§S)  costs  at  the  aircraft 
component  (WUC)  level?  _ 


List  reports/data  you  currently  receive  from  the  CSCS.  Identify  data 
elements  that  are  of  particular  interest  to  you  by  their  titles 
on  the  report.  _ 


Response 

Counts 


Do  the  reports  you  currently  receive  provide  all  the  information  you 
require?  Circle  one.  (Y)  (N) 

Indicate  information  you  would  like  to  see  from  CSCS  that  is  not 
included  in  current  reports.  _ 


6.  Do  you  consider  the  CSCS  data  to  be  useful?  Circle  one.  (Y)  (N) 

7.  Is  the  format  of  the  current  output  reports  satisfactory.  Circle  one. 

(Y)  (N) 

a.  Is  the  media  of  the  current  output  reports  (microfiche,  tape) 
satisfactory?  Circle  one.  (Y)  (N) 

8.  How  in  your  opinion  can  the  CSCS  output  reports  be  improved?  _ 


Do  you  have  a  continuing/regular  need  for  CSCS  data.  Circle  one.  (Y)  (N)  18/3 

Is  the  meaning  of  the  information  on  CSCS  reports  clear  and  understood?  15/5 

Circle  one.  (Y)  (N) 

a.  Explain  if  NO.  _  5 


Figure  B-l.  Questionnaire  Response  Counts 


Responses  were  few  enough,  and  varied  enough,  so  that 
statistical  treatment  is  not  warranted.  The  individual  responses 
to  questions  5,  8,  and  15  deserve  study  of  00V  personnel.  In 
some  cases  users  need  an  explanation.  In  others,  suggestions  should 
be  considered  for  some  future  modification  of  the  CSCS. 

Information  Spectrum  has  noted  the  following  from  its  review 
of  the  responses: 

(1)  Training  Packages 

Question  11  shows  a  strong  desire  for  a  training  package  in 
the  derivation/definition  of  CSCS  data.  It  is  interesting  to  note 
the  following  by  comparing  answers  to  questions  la,  10,  and  11: 

Of  the  22  who  desire  a  training  package,  13  said  the  meaning  of 
information  on  the  reports  is  clear  and  understood,  16  said  the 
User's  Manual  provides  an  adequate  description,  and  9  of  these  said 
both.  We  interpret  these  answers  to  mean  that  many  users  think  they 
understand  the  outputs,  but  are  not  sure.  Several  users  raised 
questions  or  made  comments  about  CSCS  outputs  which  indicated  to  us 
that  they  are  less  clear  to  the  careful  analyst  than  to  the  casual 
user.  This  is  borne  out  by  ISI's  own  experience.  Even  members  of 
the  OOV  staff  have  expressed  uncertainty  about  how  some  outputs  are 
calculated. 

Information  Spectrum  recommends  the  development  of  a  training 
package.  It  should  include  a  much  more  careful  description  of  the 
algorithms  and  of  the  associated  data  processing.  It  should  identify 
the  meaning  of  each  element  of  each  output  report.  Whenever  possible, 
these  meanings  should  be  expressed  in  terms  of  calculation  procedures. 
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The  training  package  should  also  include  a  set  of  representative 
analyses  based  on  CSCS  outputs . 

(2)  Output  Quality 

Most  of  the  respondents  consider  the  outputs  useful,  timely, 
and  satisfactory  in  format  and  medium.  A  few  recognized  problems 
associated  with  the  state  of  development  of  the  system,  e.g.,  no 
engine  data,  meaningless  zero  outputs/15  data  missing  from  the 
NSN/WUC  Cross-Reference  Dictionary.  Several  expressed  skepticism 
about  the  current  accuracy  of  the  outputs. 

(3)  User  Interest 

While  it  may  be  argued  that  only  the  most  interested  users 
bothered  to  respond,  ISI  felt  that  the  penetrating  questions  and 
comments  on  many  questionnaires  indicated  a  sincere  interest  and 
an  impressive  depth  of  understanding  on  the  part  of  many  responders. 


(1) 


It  appears  that  the  CSCS  programs  treat  some  missing  data  as 
zeros,  which  is  ill  advised. 
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This  report  summarizes  the  activities  of  this  research  project, 
xt  complies  the  recommendations  which  arose  out  of  the  analyses 
of  the  30  individual  algorithms.  Several  recommendations  have  been 
included  which  relate  to  CSCS  as  a  whole,  rather  then  to  any 
particular  algorithm.  It  also  provides  an  analysis  of  a  survey 
of  CSCS  users. 

This  volume  presents  ISIs  conclusions  and  recommendations,  and  the 
comments  of  the  Office  of  VAMOSC. 
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